This invention relates to data storage in a computerized storage system, such as a storage area network (SAN). More particularly, the present invention relates to a new and improved technique of handling data access requests within the storage system in such a manner that the devices that issue the requests (e.g. xe2x80x9cserversxe2x80x9d) have improved flexibility in selecting the path through which to send the requests without adversely affecting the ability of the devices that receive the requests (e.g. xe2x80x9cstorage arraysxe2x80x9d) to respond to the requests.
Current prior art high-capacity computerized data storage systems, such as the one shown in FIG. 1, typically involve a storage area network (SAN) 100 within which one or more conventional storage arrays 102 store data on behalf of one or more servers 104. The servers 104 typically service data storage requirements of several client devices 106, as shown in FIG. 1. The servers 104 are typically connected through switches or SAN""s, such as Fibre Channel (FC) SAN fabrics 108, to the storage arrays 102. The servers 104 access a plurality of logical volumes 110 present on the storage arrays 102 on behalf of the client devices 106.
Each storage array 102 typically includes a bank 112 of individual storage devices (not shown, e.g. hard drives, compact disk (CD) drives, tape drives, etc.), typically arranged in a RAID (Redundant Array of Independent Drives) configuration. The RAID storage devices supply data storage space for the logical volumes 110. The logical volumes 110 are commonly striped across multiple storage devices in the banks 112 of storage devices, and may be striped across multiple storage arrays 102. The servers 104 that access a given logical volume 110 must have a striping definition for the logical volume 110 if the logical volume 110 is striped across multiple storage arrays 102 and must have a connection, or path, to the storage array 102 that contains the logical volume 110, or a portion thereof. A manager device 113 typically sets up the logical volumes 110 and monitors for problems, such as a storage device that has failed or is about to fail. Through a discovery process, the servers 104 typically discover the logical volumes 110 on the storage arrays 102 and the array controllers 114 through which the servers 104 can access the logical volumes 110. The servers 104 are thus configured to use the discovered logical volumes 110.
Each storage array 102 also commonly includes more than one array controller 114, through which the storage devices and logical volumes 110 are accessed. Each array controller 114 typically connects to one of the switched fabrics 108. Thus, a data transfer path between one of the servers 104 and one of the array controllers 114 in one of the storage arrays 102 is established from a host bus adapter (HBA) 116 in the server 104, through the switched fabric 108 (to which the host bus adapter 116 is attached), and to the array controller 114 (to which the switched fabric 108 is attached).
Some of the servers 104 may access the same logical volume 110 through more than one data transfer path through the switched fabrics 108 on behalf of the same or different client devices 106. Therefore, more than one array controller 114 of a storage array 102 may receive a data access request to the same logical volume 110, or portion thereof, present on the storage array 102.
When one of the array controllers 114 of a given storage array 102 processes a data access request to a given logical volume 110, that array controller 114 is said to have access control or xe2x80x9cownershipxe2x80x9d of the logical volume 110. When one array controller 114 has ownership of the logical volume 110, no other array controller 114 in the storage array 102 can access the logical volume 110 without transferring ownership to itself, due to cache coherency issues.
When an array controller 114 receives a data access request to a logical volume 110 that the array controller 114 does not currently own, it transfers ownership of the logical volume 110 to itself in an automatic volume transfer (AVT) process and then processes the data access request. Upon transfer of ownership, the array controller 114 giving up ownership typically must xe2x80x9cflushxe2x80x9d cached data to the storage devices, so the array controller 114 that is receiving ownership will have the correct, up-to-date data in the storage devices. The time required to perform the cache flush, however, degrades the overall performance of the storage array 102. Additionally, the data access request issued by the server 104 may xe2x80x9ctimeoutxe2x80x9d causing the server 104 to erroneously determine that the array controller 114 is not operating if the cache flush takes too much time. Furthermore, when the same logical volume 110 is repeatedly accessed through different array controllers 114, then the array controllers 114 repetitively transfer ownership of the logical volume 110 back and forth between themselves. The repetitive ownership transferring is called xe2x80x9cthrashing.xe2x80x9d Thrashing can severely degrade the performance of data accesses to the affected logical volume 110, since significant time is taken up performing the AVT processes instead of accessing the affected logical volume 110.
When the servers 104 access the same logical volume 110 through the same common array controller 114, then thrashing and unnecessary AVT processes are avoided, because the common array controller 114 can process all of the data access requests. However, it is sometimes advantageous for one or more of the servers 104 to switch between its host bus adapters 116 for accessing the same logical volume 110, even when no array controller 114 has failed. In this manner, the servers 104 can optimize the use of their host bus adapters 116, but at the expense of thrashing between the array controllers 114, since the host bus adapters 116 are connected to the storage array 102 through different data transfer paths.
Additionally, the servers 104 must be able to discover the logical volumes 110 to be able to configure themselves and to route the data access requests to the appropriate array controllers 114. As the size of the SAN 100 and the number of the storage arrays 102 increases, however, the complexity of the tasks of the servers 104 increases significantly, such that maintaining and occasionally reconfiguring the complete description or striping definition of the logical volumes 110 in the servers 104 is extremely burdensome.
Furthermore, whenever a new storage array 102 or storage device is added to (or deleted from) the SAN 100 or whenever the distribution of a logical volume 110 across the banks 112 of storage devices or across the storage arrays 102 is changed, the servers 104 must be reconfigured to handle the changes. The reconfiguration takes time away from handling data access requests by the servers and becomes considerably more complex as the number of storage arrays 102 increases.
It is with respect to these and other background considerations that the present invention has evolved.
The present invention enables servers in a storage system to issue data access requests through different data transfer paths without causing ownership of the logical volumes to thrash back and forth between controllers of a storage array. Thus, the flexibility of the performance of the servers is increased without sacrificing the performance of the controllers. Additionally, particularly in situations where logical volumes are striped across multiple storage arrays, the servers are relieved of some of the burden of maintaining a description or striping definition of the logical volumes, so reconfiguring and scaling the storage system are simpler and easier.
In accordance with these features, routing capabilities are included in I/O (input/output) modules that control access to the storage devices. Alternatively, the routing capabilities are integrated into switched fabrics between the servers and the storage arrays, instead of into the I/O modules. The routing capabilities are also preferably included in other types of devices (e.g. intelligent storage hubs, network attached storage appliances, snapshot/backup appliances, remote data appliances, etc.) that control access to the storage arrays.
The I/O modules (or other routing-capability devices) receive the data access requests from the servers and, independently of the data transfer path through which the servers sent the requests, the I/O modules route the data access requests to the appropriate switched fabrics and to the appropriate controllers in the storage arrays. In this manner, when applicable, the servers are relieved of the burden of managing a complete description of the logical volumes (e.g. storage array ID, volume ID, logical unit number, striping order, striping depth, etc.). Instead, the servers are preferably configured only with a xe2x80x9cglobal identifierxe2x80x9d or xe2x80x9clogical unit numberxe2x80x9d (LUN) that essentially identifies the logical volume as a single volume (e.g. a xe2x80x9cvirtualxe2x80x9d volume), rather than as a combination of several volumes spread across multiple storage devices, and possibly across multiple storage arrays.
The servers are also configured with information regarding which host bus adapters and data transfer paths that the servers can use to send the data access requests to the logical volume. The servers preferably select a preferred path to a preferred I/O module when sending the data access request, but can typically switch to another path and I/O module when necessary to normalize the loading on the host bus adapters and increase the overall data transfer bandwidth. The selection of any path, however, has no effect on and is made without regard to the controller that will handle the response to the access request at each storage array.
The I/O modules are preferably configured with information by which the global identifiers are xe2x80x9cmappedxe2x80x9d to a xe2x80x9clocal identifierxe2x80x9d for the complete description of the logical volumes, so that the I/O modules can distribute the data access requests to the correct array controllers that have control over those logical volumes. The I/O modules are also preferably configured with information indicating which controller in each storage array currently has xe2x80x9cownershipxe2x80x9d of the logical data volumes or portions thereof, so that the I/O modules can route the data access requests to the current owner controller and avoid unnecessary ownership transfers, regardless of the data transfer path selected by the servers. Thus, the 110 module receives an access request directed by a server to a logical volume as a single volume whose physical location or distribution is unknown to the server. The I/O module distributes the access request to the relevant controller(s), receives back one or more responses from the controller(s), assembles the responses into a single response if necessary and sends the response to the server.